{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 一.简介\n",
    "xgboost在集成学习中占有重要的一席之位，通常在各大竞赛中作为杀器使用，同时它在工业落地上也很方便，目前针对大数据领域也有各种分布式实现版本，比如xgboost4j-spark,xgboost4j-flink等。xgboost的基础也是gbm，即梯度提升模型，它在此基础上做了进一步优化..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 二.损失函数：引入二阶项\n",
    "xgboost的损失函数构成如下，即一个经验损失项+正则损失项：   \n",
    "\n",
    "$$\n",
    "Cost(y,F_{m-1},f_m)=\\sum_{i=1}^n L(y_i,F_{m-1}(x_i)+f_m(x_i))+\\Omega(f_m)\n",
    "$$   \n",
    "\n",
    "这里$n$表示样本数，$F_{m-1}$表示前$m-1$轮模型，$f_m$表示第$m$轮新训练模型，所以$F_m=F_{m-1}+f_m$，$\\Omega(f_m)$是对第$m$轮新训练模型进行约束的正则化项，在前面第6小节做过探索，对损失函数近似做二阶泰勒展开，并对近似损失函数做优化，通常会收敛的更快更好，接下里看下对第$i$个样本的经验项损失函数做二阶展开：   \n",
    "\n",
    "$$\n",
    "L(y_i,F_{m-1}(x_i)+f_m(x_i))=L(y_i,F_{m-1}(x_i))+g_if_m(x_i)+\\frac{1}{2}h_if_m^2(x_i)\n",
    "$$   \n",
    "\n",
    "这里：   \n",
    "\n",
    "$$\n",
    "g_i=\\frac{\\partial L(y_i,F_{m-1}(x_i))}{\\partial F_{m-1}(x_i)}\\\\\n",
    "h_i=\\frac{\\partial^2 L(y_i,F_{m-1}(x_i))}{\\partial {F_{m-1}(x_i)}^2}\n",
    "$$    \n",
    "\n",
    "对于第$m$轮，$L(y_i,F_{m-1}(x_i))$为常数项，不影响优化，可以省略掉，所以损失函数可以表示为如下：   \n",
    "\n",
    "$$\n",
    "Cost(y,F_{m-1},f_m)=\\sum_{i=1}^n [g_if_m(x_i)+\\frac{1}{2}h_if_m^2(x_i)]+\\Omega(f_m)\n",
    "$$   \n",
    "\n",
    "这便是xgboost的学习框架，针对不同问题，比如回归、分类、排序，会有不同的$L(\\cdot)$以及$\\Omega(\\cdot)$，另外由于需要二阶信息，所以$L(\\cdot)$必须要能二阶可微，接下来对基学习器为决策树的情况做推导"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 三.基学习器：回归决策树\n",
    "下面推导一下基学习器为回归树的情况，当选择决策树时，它的正则化项如下：   \n",
    "\n",
    "$$\n",
    "\\Omega(f_m)=\\gamma T+\\frac{1}{2}\\lambda\\sum_{j=1}^T\\omega_j^2\n",
    "$$  \n",
    "\n",
    "其中，$j=1,2,...,T$表达对应的叶节点编号，$\\omega_j$表示落在第$j$个叶节点的样本的预测值，即：   \n",
    "\n",
    "$$\n",
    "\\omega_j=f_m(x_i),x_i\\in I_j\n",
    "$$  \n",
    "\n",
    "$I_j$表示第$j$个叶子节点所属区域，所以决策树的损失函数可以改写为如下：  \n",
    "\n",
    "$$\n",
    "Cost(y,F_{m-1},f_m)=\\sum_{j=1}^T[(\\sum_{i\\in I_j}g_i)\\omega_j+\\frac{1}{2}(\\sum_{i\\in I_j}h_i+\\lambda)\\omega_j^2]+\\gamma T\n",
    "$$  \n",
    "\n",
    "这其实是关于$\\omega$的一元二次函数，直接写出它的最优解：   \n",
    "\n",
    "$$\n",
    "\\omega_j^*=-\\frac{G_j}{H_j+\\lambda}\n",
    "$$  \n",
    "\n",
    "这里$G_j=\\sum_{i\\in I_j}g_i,H_j=\\sum_{i\\in I_j}h_i$,可见$L_2$正则项起到了缩小叶子节点权重的效果，减少其对整个预测结果的影响，从而防止过拟合，将$\\omega_j^*$带入可得损失值：   \n",
    "\n",
    "$$\n",
    "Cost(y,F_{m-1},f_m^*)=-\\frac{1}{2}\\sum_{j=1}^T\\frac{G_j^2}{H_j+\\lambda}+\\gamma T\n",
    "$$   \n",
    "\n",
    "\n",
    "#### 特征选择\n",
    "\n",
    "很显然，上面的损失函数可以直接用于特征选择中，对某节点在分裂前的评分为：  \n",
    "\n",
    "$$\n",
    "Score_{pre}=-\\frac{1}{2}\\frac{G^2}{H+\\lambda}+\\gamma\n",
    "$$  \n",
    "\n",
    "分裂后，左右子节点的评分和为：  \n",
    "\n",
    "$$\n",
    "Score_{pre}=-\\frac{1}{2}(\\frac{G_L^2}{H_L+\\lambda}+\\frac{G_R^2}{H_R+\\lambda})+2\\gamma\n",
    "$$  \n",
    "\n",
    "所以分裂所能带来的增益：  \n",
    "\n",
    "$$\n",
    "Score=\\frac{1}{2}[\\frac{G_L^2}{H_L+\\lambda}+\\frac{G_R^2}{H_R+\\lambda}-\\frac{G^2}{H+\\lambda}]-\\gamma\n",
    "$$  \n",
    "\n",
    "这里$G=G_L+G_R,H=H_L+H_R$   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 四.代码实现\n",
    "\n",
    "这部分对xgboost中的回归树做简单实现，大体流程其实与CART回归树差不多，下面说下它与CART回归树不一样的几个点：   \n",
    "\n",
    "（1）这里`fit`与之前的CART回归树有些不一样了，之前是`fit(x,y)`，而现在需要`fit(x,g,h)`；  \n",
    "\n",
    "（2）特征选择不一样了，之前是求平方误差的增益，现在需要利用一阶和二阶导数信息，见上面的$Score$   \n",
    "\n",
    "（3）叶子节点的预测值不一样了，之前是求均值，现在需利用一阶和二阶导数信息，见上面的$w_j^*$  \n",
    "\n",
    "接下来对xgboost所需要用到的回归树做简单实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.chdir('../')\n",
    "import numpy as np\n",
    "from ml_models.wrapper_models import DataBinWrapper\n",
    "\n",
    "\"\"\"\n",
    "xgboost基模型：回归树的实现，封装到ml_models.ensemble\n",
    "\"\"\"\n",
    "\n",
    "class XGBoostBaseTree(object):\n",
    "    class Node(object):\n",
    "        \"\"\"\n",
    "        树节点，用于存储节点信息以及关联子节点\n",
    "        \"\"\"\n",
    "\n",
    "        def __init__(self, feature_index: int = None, feature_value=None, y_hat=None, score=None,\n",
    "                     left_child_node=None, right_child_node=None, num_sample: int = None):\n",
    "            \"\"\"\n",
    "            :param feature_index: 特征id\n",
    "            :param feature_value: 特征取值\n",
    "            :param y_hat: 预测值\n",
    "            :param score: 损失函数值\n",
    "            :param left_child_node: 左孩子结点\n",
    "            :param right_child_node: 右孩子结点\n",
    "            :param num_sample:样本量\n",
    "            \"\"\"\n",
    "            self.feature_index = feature_index\n",
    "            self.feature_value = feature_value\n",
    "            self.y_hat = y_hat\n",
    "            self.score = score\n",
    "            self.left_child_node = left_child_node\n",
    "            self.right_child_node = right_child_node\n",
    "            self.num_sample = num_sample\n",
    "\n",
    "    def __init__(self, max_depth=None, min_samples_split=2, min_samples_leaf=1, gamma=1e-2, lamb=1e-1,\n",
    "                 max_bins=10):\n",
    "        \"\"\"\n",
    "        :param max_depth:树的最大深度\n",
    "        :param min_samples_split:当对一个内部结点划分时，要求该结点上的最小样本数，默认为2\n",
    "        :param min_samples_leaf:设置叶子结点上的最小样本数，默认为1\n",
    "        :param gamma:即损失函数中的gamma\n",
    "        :param lamb:即损失函数中lambda\n",
    "        \"\"\"\n",
    "        self.max_depth = max_depth\n",
    "        self.min_samples_split = min_samples_split\n",
    "        self.min_samples_leaf = min_samples_leaf\n",
    "        self.gamma = gamma\n",
    "        self.lamb = lamb\n",
    "\n",
    "        self.root_node: self.Node = None\n",
    "        self.dbw = DataBinWrapper(max_bins=max_bins)\n",
    "\n",
    "    def _score(self, g, h):\n",
    "        \"\"\"\n",
    "        计算损失损失评分\n",
    "        :param g:一阶导数\n",
    "        :param h: 二阶导数\n",
    "        :return:\n",
    "        \"\"\"\n",
    "        G = np.sum(g)\n",
    "        H = np.sum(h)\n",
    "        return -0.5 * G ** 2 / (H + self.lamb) + self.gamma\n",
    "\n",
    "    def _build_tree(self, current_depth, current_node: Node, x, g, h):\n",
    "        \"\"\"\n",
    "        递归进行特征选择，构建树\n",
    "        :param x:\n",
    "        :param y:\n",
    "        :param sample_weight:\n",
    "        :return:\n",
    "        \"\"\"\n",
    "        rows, cols = x.shape\n",
    "        # 计算G和H\n",
    "        G = np.sum(g)\n",
    "        H = np.sum(h)\n",
    "        # 计算当前的预测值\n",
    "        current_node.y_hat = -1 * G / (H + self.lamb)\n",
    "        current_node.num_sample = rows\n",
    "        # 判断停止切分的条件\n",
    "        current_node.score = self._score(g, h)\n",
    "\n",
    "        if rows < self.min_samples_split:\n",
    "            return\n",
    "\n",
    "        if self.max_depth is not None and current_depth > self.max_depth:\n",
    "            return\n",
    "\n",
    "        # 寻找最佳的特征以及取值\n",
    "        best_index = None\n",
    "        best_index_value = None\n",
    "        best_criterion_value = 0\n",
    "        for index in range(0, cols):\n",
    "            for index_value in sorted(set(x[:, index])):\n",
    "                left_indices = np.where(x[:, index] <= index_value)\n",
    "                right_indices = np.where(x[:, index] > index_value)\n",
    "                criterion_value = current_node.score - self._score(g[left_indices], h[left_indices]) - self._score(\n",
    "                    g[right_indices], h[right_indices])\n",
    "                if criterion_value > best_criterion_value:\n",
    "                    best_criterion_value = criterion_value\n",
    "                    best_index = index\n",
    "                    best_index_value = index_value\n",
    "\n",
    "        # 如果减少不够则停止\n",
    "        if best_index is None:\n",
    "            return\n",
    "        # 切分\n",
    "        current_node.feature_index = best_index\n",
    "        current_node.feature_value = best_index_value\n",
    "        selected_x = x[:, best_index]\n",
    "\n",
    "        # 创建左孩子结点\n",
    "        left_selected_index = np.where(selected_x <= best_index_value)\n",
    "        # 如果切分后的点太少，以至于都不能做叶子节点，则停止分割\n",
    "        if len(left_selected_index[0]) >= self.min_samples_leaf:\n",
    "            left_child_node = self.Node()\n",
    "            current_node.left_child_node = left_child_node\n",
    "            self._build_tree(current_depth + 1, left_child_node, x[left_selected_index], g[left_selected_index],\n",
    "                             h[left_selected_index])\n",
    "        # 创建右孩子结点\n",
    "        right_selected_index = np.where(selected_x > best_index_value)\n",
    "        # 如果切分后的点太少，以至于都不能做叶子节点，则停止分割\n",
    "        if len(right_selected_index[0]) >= self.min_samples_leaf:\n",
    "            right_child_node = self.Node()\n",
    "            current_node.right_child_node = right_child_node\n",
    "            self._build_tree(current_depth + 1, right_child_node, x[right_selected_index], g[right_selected_index],\n",
    "                             h[right_selected_index])\n",
    "\n",
    "    def fit(self, x, g, h):\n",
    "        # 构建空的根节点\n",
    "        self.root_node = self.Node()\n",
    "\n",
    "        # 对x分箱\n",
    "        self.dbw.fit(x)\n",
    "\n",
    "        # 递归构建树\n",
    "        self._build_tree(1, self.root_node, self.dbw.transform(x), g, h)\n",
    "\n",
    "    # 检索叶子节点的结果\n",
    "    def _search_node(self, current_node: Node, x):\n",
    "        if current_node.left_child_node is not None and x[current_node.feature_index] <= current_node.feature_value:\n",
    "            return self._search_node(current_node.left_child_node, x)\n",
    "        elif current_node.right_child_node is not None and x[current_node.feature_index] > current_node.feature_value:\n",
    "            return self._search_node(current_node.right_child_node, x)\n",
    "        else:\n",
    "            return current_node.y_hat\n",
    "\n",
    "    def predict(self, x):\n",
    "        # 计算结果\n",
    "        x = self.dbw.transform(x)\n",
    "        rows = x.shape[0]\n",
    "        results = []\n",
    "        for row in range(0, rows):\n",
    "            results.append(self._search_node(self.root_node, x[row]))\n",
    "        return np.asarray(results)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面简单测试一下功能，假设$F_0(x)=0$，损失函数为平方误差的情况，则其一阶导为$g=F_0(x)-y=-y$，二阶导为$h=1$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#构造数据\n",
    "data = np.linspace(1, 10, num=100)\n",
    "target1 = 3*data[:50] + np.random.random(size=50)*3#添加噪声\n",
    "target2 = 3*data[50:] + np.random.random(size=50)*10#添加噪声\n",
    "target=np.concatenate([target1,target2])\n",
    "data = data.reshape((-1, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1d8fa8fd828>]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X90XOV95/H315JsyWBbNhbGlqE2CTEEnNigAKk2LTFkISE/BE3Spg2hPbTunk1aYBOo4ZwttE0OziENTTab7HFDCjlliTlAgCW/i6EJ3q0TGQsM2IbwIwbJtmSDsIllI8nf/WNmpNHozsydH3fmzszndY6ORnfu3Ptojv2dR8/zfb6PuTsiIlL7ZlS7ASIiUh4K6CIidUIBXUSkTiigi4jUCQV0EZE6oYAuIlInQgd0M2sys21m9nDy5+VmtsXMnjezjWY2M7pmiohIPoX00K8GdqT9/GXgNnc/DXgduKqcDRMRkcKECuhmthS4FPh28mcD1gD3Jk+5E+iJooEiIhJOc8jz/gm4HpiT/PkEYNjdx5I/vwp0Br3QzNYCawGOO+64c04//fTiWysi0oC2bt2639078p2XN6Cb2YeBQXffamYXpA4HnBpYQ8DdNwAbALq6ury3tzffLUVEJI2Z/SbMeWF66N3AR83sQ0ArMJdEj73dzJqTvfSlwECxjRURkdLlHUN39xvcfam7LwP+CNjk7n8CPAp8PHnalcCDkbVSRETyKiUP/W+A/2ZmvyYxpn57eZokIiLFCDspCoC7PwY8lnz8InBu+ZskIiLF0EpREZE6UVAPXUREEh7Y1s+tP9nFwPAIS9rbuO7iFfSsDszerhgFdBGRAj2wrZ8b7t/OyOg4AP3DI9xw/3aAqgZ1DbmIiBTo1p/smgjmKSOj49z6k11ValGCArqISIEGhkcKOl4pCugiIgVa0t5W0PFKUUAXESnQdRevoK2lacqxtpYmrrt4RZValKBJURGRAqUmPpXlIiJSB3pWd1Y9gGdSQBcRKaNq5qcroItI3atUkK12fromRUWkrqWCbP/wCM5kkH1gW3/Z71Xt/HQFdBGpa5UMstXOT1dAF5G6VskgW+38dAV0EalrlQyy1c5PV0AXkbpWySDbs7qTWy5fSWd7GwZ0trdxy+UrleUiIlIOlV4EVM38dAV0Eal7cVwEFAUFdBGREsVlswsFdBGRElR7MVG6vJOiZtZqZr80syfN7Bkz+7vk8TvM7CUz60t+rYq+uSIi8VLtxUTpwvTQjwJr3P1NM2sBHjezHyWfu87d742ueSIi8VbtxUTp8vbQPeHN5I8tyS+PtFUiIjWi2ouJ0oXKQzezJjPrAwaBn7n7luRTXzKzp8zsNjObleW1a82s18x6h4aGytRsEZF4qPZionTmHr6zbWbtwPeBvwIOAHuBmcAG4AV3//tcr+/q6vLe3t7iWysidS0u2SKFirrdZrbV3bvynVdQlou7D5vZY8Al7v6V5OGjZvYvwBcKb6aISEKcskUKFZc89zBZLh3Jnjlm1gZcBOw0s8XJYwb0AE9H2VARqW9xyhapVWF66IuBO82sicQHwD3u/rCZbTKzDsCAPuC/RNhOEalzccoWqVV5A7q7PwWsDji+JpIWiUhDWtLeRn9A8K5GtkitUrVFEYmFOGWLlMsD2/rpXr+J5et+QPf6TZHskpROS/9FJBYqVRUxW0ZKmEyVQrJZqjHJW1DaYqmUtigi1ZQZZCHxV8AfnNPJfVv7px1Pr2We7bXZ6p13r98UOITU2d7G5nWFjViHTVvUkIuINIxsmTT/+h+782bYFJqFU41JXgV0EWkYhQbT9PMLDdDVKAmggC4iDaPQYJp+fqEBuhqTvAroItIwgoJsNpnBt9AAXY39RZXlIiINIz2TJmjCMqUzIIOlmCycSpcEUJaLiDSkQrNWqimS4lwiIvWiUnnvlaSALiLV8eST8KtfVbUJPUDPQmBh8sDWF2FrBDcyg6uuiuDCUymgi0hZhV5NecUVsH175RtYDc3NCugiUlsKWu4+MACf+Qx86Usl3/fHT+9h/Y92cmT02MSx1pYZrPvg6Vxy1uKSr18rFNBFpGxyraacEtDHxuC112DZMli6tKB7BP0FcGvfIfrbFkBGSvg/9B3ikkveU/DvUas7Jymgi0hWhQa20KspDxwAdzjxxILbE/QXQOaHSL72FHqPazf2cc3GvsB0xjjRwiIRCZQKbP3DIziTwTNXCdjQqylTG8YXGNCz/QXQZFZQewq9Ryq5O8x7UE0K6CJ1rtia3MVsCRd6NeXgYOJ7R0eotqRk63GPu5dtmX2+Xn2ct8VTQBepY8X0slOKqRYYerl7KqAX2EPP1uNO3accy+zD9Orjui2extBF6ljoScoAxW4JF2q5e2rIpcAe+nUXrwhc3Zka1y7H2HbQPTLFdVs89dBF6lgpNbkjrRY4OAgzZsCCBQW9rBIFr9LvAZA5Oh/nbfHUQxepY6VsvBzp0vihIVi4EJrCVT7MbFfUWSbp96ilFMa8Ad3MWoGfA7OS59/r7jeZ2XLge8AC4AngCnd/K8rGikhhcg1RhBFZ8BwcnBhuiXvArHTFxFKEGXI5Cqxx93cDq4BLzOx84MvAbe5+GvA6EP26VhEpSDVqcocyOAgnnljSpK1Ml7eH7on6um8mf2xJfjmwBvjj5PE7gZuBb5W/iSJSirj0MNN74v++6zfMOPvskiZtZbpQk6Jm1mRmfcAg8DPgBWDY3ceSp7wKBL77ZrbWzHrNrHcoNbMtIg0lsyc+79DrPPaaZ91kIq5pgXEXKqC7+7i7rwKWAucCZwSdluW1G9y9y927OgpMURKR+pDeE28ZH2Xe0d+yr3Vu1vPjmhYYdwVlubj7sJk9BpwPtJtZc7KXvhQYiKB9InWvkpOC1ZqATO9xzz98EIDXZs8LPLeQSdu4T6hWWpgslw5gNBnM24CLSEyIPgp8nESmy5XAg1E2VKQeFVRuNgb3KjaApqdPLjz8BgD7Z7dPO6+Q4le5fh+or52IwgrTQ18M3GlmTSSGaO5x94fN7Fnge2b2RWAbcHuE7RSpG+lBcYYZ4xn7+kY1KZhtAvKajX3c+pNdeYNeKR8I6emTC5IB/cBxU3voBmxet6bk3+fmh57h6NixinxIxk2YLJengNUBx18kMZ4uInmkgnj/8AjG5IRTZjBPiWJSMNc1wwS9UjJS0hcpnXB4GIDX2qYG9ELHzbP9PsMjo9OONUrmjJb+i0QsPcMDsmQPZIhiUjDfNfNVESyljAAkgvrmdWv48xVzANh/3OSQSzHL6cv1AVBPFNBFIhbUs80lqlohQbVZMuUKeqFrneexcuZRjjU3M2fRwtCLnYJKAGerNTN/dktZ2lmLVMtFJGJheoZNZhxzL2kCL9+EZfqwR7b871xBr9QyAhOGhpjR0cHmGy4MdXq2sftbLl/JLZevnPY7A+VpZw1SQBeJWLYCWSltLU0lL8cPO2GZWjWaeX6qHbmCXtmKdSWX/YeVa+x+87o1Ocf8leUiImUV1LNNTYyWa4/KQicsiw3OZSkjkFaYK4xiN9pohACeSQFdJGKRlqFNyhbc+odH6F6/KfB+VQt6Q0Nw6qmhTy+lBHCjUUAXKUKhC2yiDp65hnVil4dd4JBL2cbuG4CyXEQKFMeSr/kyWGKzsfGRI3DoUEFDLrEtARxD6qGLFKjaJV9z/XWQK4MlFnnYqYqrBW4O3ahj4oVSQBcpUKkLbEqRL5ulZ3Un3es3xXfMeXAw8V2VVyOhIReRApVrgU0xcv11kBLp5s6lKrKHLuEooIsUqJoBM8xfB7Eec0710BXQI6EhF5ECVSINMZuwKXyVGHMuqpRuqoeuIZdIKKCLFKFak3RxSeErupTu4CDMmgVz5lSimQ1HAV0kpDjsjlPNvw7SFZ3pk8pBN4u4hY1JAV0khEruLJRPHFL4is70GRoKPdwShw/QWqOALpJy7Fhi4UuAr/+fJ/HDI7SmHfPRxPGeFfMr074YWT7bGHgjYCx/XhscPpz9hXv3hpoQjdMHaC1RQBdJ+cAHYNOmwKeCjybdFElrYq2k9+NP/zTv9au9eKtWKaCLpDz9NLz3vdDTM+2p//norzl4ZPrWZnNbW/js+99eidbFztP9b/Dvzw1x6Mgorc1NYHBkdJw5rS38/js6OKtzXvALL7ss77WruXirlimgi6S88Qa8731w/fXTnur8QD/fCMguueXyldCgPcazkl9BtdW/0dLELR8oPvddFRaLkzegm9nJwHeBk4BjwAZ3/5qZ3Qz8BZBMLOVGd/9hVA0VidTRo4mvuXPz1kqJ0yRdHCYO861eLaZ9cUnPrDVheuhjwOfd/QkzmwNsNbOfJZ+7zd2/El3zRCrk0CEAnjzoeWulxEVcJg5z1WIvtn1x/QCNu7wB3d33AHuSjw+Z2Q5A76rUl4MHAXjopTcZObU2JuPiMnGYbXikyayk9sXtA7QWFFTLxcyWAauBLclDnzOzp8zsO2YWmLtlZmvNrNfMeodSy35F4uaNNwB4ZSy4jxPHybioJg4f2NZP9/pNLF/3A7rXb8pb5z1bbZtx90jaJ9mFDuhmdjxwH3CNux8EvgW8DVhFogf/j0Gvc/cN7t7l7l0dqt8gcZXsoc86ITinPI6TcVFUfSxm845sxcA6q1iVslGFynIxsxYSwfwud78fwN33pT3/z8DDkbRQpBKSAf0PLjiDf9s5o6DJuGpNTEYxcVjsME624RFNbFZW3h66mRlwO7DD3b+adnxx2mmXAU+Xv3kiFZIM6Bece1pBpWeruR1dFGVyyzmME+syvnUqTA+9G7gC2G5mfcljNwKfMrNVgAMvA38ZSQtFKiE5hs7cufSctih00Kn2xGS5Jw7Lnf+tic3KCpPl8jgQVBpNOedSP5I9dObOLehl9baiUfnftU07FolAIqA3N0Nra/5z01RzO7ooaJiktmnpvwgkAvq8eQXX6Q7q0RqJsfRVf/dTzGD48GhNLYzRMEntUkAXgcQYeoHDLTB1RWP/8AhGYlIJYHhkspiXyr9KJWjIRQQSPfQiAjokAvTmdWvobG8jeClNQnp9k7gpdDGRxJN66CJQUkBPCTMRGsfJ0rjUhJHSqYcuAmUJ6GEmQvOdU42ecr5qiVI7FNBFYHJStARBNU3ShVlxWo1FSvWWetnIFNBFoOhJ0XSZKX/tbS3Mn90SOv2vWj3leku9bGQaQxeBsgy5QGkpf9XqKWsxUf1QD10kbbeiaqpWT1mLieqHeugiyd2KSh1DL1WuRUrd6zdFujBJi4nqgwK6SFphrmrKtUhJqYQShoZcRIoszBWFXIuUlEoo+Sigi8QooKcolVCKoYAuEsOArlRCKYYCukhqDL3Kk6Lpsm28rFRCyUWToiIx7KGnT5BWeq9SqV0K6CIxDOigVEIpnAK6SJG7FVXSA9v61VuXvBTQRd54o6jdiipF5W0lrLyTomZ2spk9amY7zOwZM7s6eXyBmf3MzJ5Pfp8ffXNFIlCmOi5RUXlbCStMlssY8Hl3PwM4H/ismb0TWAc84u6nAY8kfxapPTEP6MpJl7DyBnR33+PuTyQfHwJ2AJ3Ax4A7k6fdCfRE1UiRSMU8oCsnXcIqKA/dzJYBq4EtwCJ33wOJoA+cmOU1a82s18x6h4aGSmutSBTKUAs9SspJl7BCB3QzOx64D7jG3Q+GfZ27b3D3Lnfv6ujoKKaNItEqw25FUVJ5WwkrVJaLmbWQCOZ3ufv9ycP7zGyxu+8xs8XAYFSNFIlCKhXwwT37+cULh7Bt/bENkspJlzDCZLkYcDuww92/mvbUQ8CVycdXAg+Wv3ki0Ujfv3PO0cPsY2ZF9u8UiVKYHno3cAWw3cz6ksduBNYD95jZVcBu4BPRNFGk/FKpgDPHRpk1PsqhmbMnUgHTe8Ja0CO1JG9Ad/fHSWycEuTC8jZHpDJSKX/Hv3UYgDdnzQam7g4EaEGP1BStFJWGtKS9jf7hEY4/mgjoh5IBHSYDd2vLjKwLehTQJY4U0KWhpIZQUlu8zUn10GfOnnLeyOj4tGCeogU9ElcK6NIwMmuiODAnoIeejxb0SFxpgwtpGEE1UVJDLpk9dID2thYt6JGaoh66NIygoZLUpOjo8XOmHG9raeLmj54JaJMJqR0K6NIwUhOh6VI99KsvO4cv9r4WGLgVwKVWKKBLw7ju4hVTxtAB5o8dAeCD3Sv44EUaG5fapoAuDSNon86PLJsd+92KRMJSQJealWsVZ7bnptVE+ex9iUqLMd2tSKQQCuhSk3JtywYFrPCMeaVFkUIooEtNyrctW+gVnjHf3EKkEAroUpOK2ZYt8LmYb24hUggtLJKalG215gwzvJDXqIcudUQBXWpS0LZsAOMeHM6NyUqKU2qeawxd6oiGXCRevv99+Nu/hSyBOaUHWHNkjP1vHmV0/BiG4Vn75lPN+KpxcG4rc1ub4eWX4aKLSm+3SAwooEu83HEHDAzA+98/cah/eITn9h3iyFvjtM5s4h2L5tDZ3sZcIDVY8qPte0KG84RXZzbx/hUnwhlnwBVXlPEXEKkeBXSpiFA7/7jDli1w6aXw3e9OvC5zdWdbS9O0TZK/uH7TtGX9uRjw0vpLS/qdROJGY+gSufT9O53JvPBp+3fu3g379sF55/HAtn6612/imo19OdMTU4LG1Ntampg/uyWwTSqBK/VIAV0ily9nfMKWLQA81r584gMgm8wUxJ7Vndxy+Uo629swoLO9jVsuX8lNHzlTJXClYWjIRSKTvjtQkIHhkSlDMV/6v/fyhzNncdNLTYyMjuW8dlAPe9qy/jQqgSuNwDxPNoGZfQf4MDDo7mclj90M/AUwlDztRnf/Yb6bdXV1eW9vb0kNltoQNPadqb2thaNjxybOueeu62nBuexPbs15bSOx21CngrM0CDPb6u5d+c4LM+RyB3BJwPHb3H1V8itvMJfGEjTMkq6tpQmzySX6zeNjrNz7Ak+c9I6c100Fc8gxFi/SoPIGdHf/OfBaBdoidSTXEvzU+Pbw4dGJYyv2/4a2saM8uTg4oKcmODP/ngwcixdpUKVMin7OzJ4ys++Y2fxsJ5nZWjPrNbPeoaGhbKdJnUhlp2QbyOtsb2PzujX0rO6cMg6+aiARlLctmT5ZGfQBkC7Xh4dIIyk2oH8LeBuwCtgD/GO2E919g7t3uXtXR0dHkbeTWpCenhgkM7skPdXw3Xue40DbXF6Zt2jKawwCPwDSKQVRJKGogO7u+9x93N2PAf8MnFveZkktyjVunuplp09gpqcarhp4jr4lK6ZtNJEerLPlmisFUSShqLRFM1vs7nuSP14GPF2+JklcpacYzmtrwQyGD49OpAJmG/pI9bKD9KzupOdtc/AbX+HHZ/3+lOcyg3XQFnLKchGZlDegm9ndwAXAQjN7FbgJuMDMVpFIOHgZ+MsI2ygxkJmGODwyOZ6dyjZpn93C6wHj3JlDIpllAL7cPsR/cqfrExfTeagtZ7DOlWsu0ujyBnR3/1TA4dsjaIvExegofP3r8OabE4f2/+Il1h4JnpRMaW1pYuyYMzZ+bOJYc9MMLjrjRPi7XwCwY89BXtkxyMfTzjm29zkAfvePP8Tm+Vnn10UkD60UlekeeQS+8IUph/68lOv9fPLhGcmvTE8sfzdnK5iLlEQBXaZ75pnE96EhOOEEALpDVDNMpSTmsnzdDwJTGg14qYimisgkFeeSaX7zi1/x2vHtLP/KFrq//CgP9A1w3SWn0zazOZGFEvDVNrOZ6y45Pevzqa8l82dnPy4iJVFAlyke2NbPa1ufYteCk6eUugWmVDNsb2th/uyWKZUNw0xWKvVQJDoacpEpbv3xTn40tJsH3zmZQphaXp9a4JMplbVy7ca+vKmESj0UiY4Cukwx9mo/c4/+ludPOHnK8Ww55pnpjOk9+lxBXQFcpPw05CJTnH90HwDPLzxlynEnMTGaWdkw9OYVIhI5BXSZ4qoTjgDw6xNOmfZcULnabD13FcwSqTwFdJniXQcHeGvuPGZ2Lg58PrP3rYJZIvGhgF7DUqVql6/7QeBwSFGefZaZZ53J5hsuxLKckt77VtaKSHwooNeo9FK16emFJQf1Z5+Fd74TCNf7zrY5syY9RSpPWS41KtdkZKHBNJV2eGRgL1v372f73E5Wkuh9Z+4LGtT7VtaKSDwooNeock1GpqcdnnvgFQC+tqeFD2/rV864SI1RQK9RS9rbAmurFDoZmd7TP23/bgCebV/CjmRPX71vkdqhgF5jUsMj/cMjGEwpdJVvMjKzDnnmphRvP/AKb85sY2BOB6a0Q5Gao4AeU0HBF5gypu0wEdQ704ZDwry2f3iEazf2TflAOG3/bn59wtJEsSylHYrUHAX0GMq2nL61Zca0idBUME+VrS30tenefuAVHl+2WmmHIjVKAT2GsmWwZNuAOX3YpNDXpsw98iYnvfka+5aeqrRDkRqlgB5DqQD915vv5l17nst7fmtLEzzzvwD4+x37AjeQyOf4txL3/Ox//QgomIvUJAX0GFrS3sbwvgNcvflu9h2/gAOz5wHQNMNwh2M+GbJnmNHZ3gYDbwHQOXKI0bT9Ogux9dRVnNPdXfovICJVkTegm9l3gA8Dg+5+VvLYAmAjsAx4Gfiku78eXTMbQ3oGy3v3/pomP8aNF3+Ox97WRVtLE7dcvhKYnhd+ZlqPemfGGHo2QRkyt1y+knMWLIjgNxORSgjTQ78D+Abw3bRj64BH3H29ma1L/vw35W9e/QjKPEkfp86czFw9sBOAviXvmJLBAsF1xtOvP6+thdaWGbx+eDSwLanracGQSH3JG9Dd/edmtizj8MeAC5KP7wQeQwE9qzCbQGROZq7a8xwvLOjkuMWLJoJvth2BMq8/PDJKW0sTnz7/FO7b2h+4dF8LhkTqT7HFuRa5+x6A5PcTs51oZmvNrNfMeoeGhoq8XW0LswnElCX77qwe2EnfkhUTwT9XEa5s139055AKZ4k0kMgnRd19A7ABoKurq5gEjJoXpu5K+lL+pQcH6fjtMNuWnE6TWd4iXLmur564SOMotoe+z8wWAyS/D5avSfUnTBna9Lriq/sT4+fPnnIG4x78GZj5YVDIfUWkPhUb0B8Crkw+vhJ4sDzNia9SNpPItQlE6rrXbuxjVvMM5s9uYfXALo60zOLKtR9JpCQGyPZhkHl9EWkcYdIW7yYxAbrQzF4FbgLWA/eY2VXAbuATUTay2orZ2T5dtjK0QOBkZs/RV2h973l87D2/gzc3561JrjK3IgLhslw+leWpC8vcltjKN6kZJpAGjWV3r9807brjI0c4fsfT8PlrJ14X5h4aKxcRrRQNIdukY6qnXmzPPei6Z+57gZnjo3D++RPHFKxFJAztKRpCtsnFXBkoxV539UDyteedV1gjRaThKaCTf8Iz26RjrgyUMJOoQdft2ruLwyd1wpIlJf5WItJoGn7IJe+E56OP0vPNb/Ke4RF27T3EkdFxWluaWHHSHHbtPRRYM2Vm0wzGH3JuPDYZ8Ju+a/R3zpuStdID0677vt19tHzokmh/aRGpSw0f0LNNeF6zsY+bH3qG/3HXf+ecl5/krfkn0XX8LOa2NsMxYGA/c46MMXjwCGlxmxkGZhBU8PCtAwYLj5tyrDP5RROJ6y47Ba64ory/pIg0hIYP6NkmPAGGD7/Fma/u5OHT38f1H7pmoiJhaoJyLrApoOhW5tZuKQa8tP7SvIW6RESK0fABPX3JfaZThveyYOQgTy5+BzB9yT0EZ6CkSuAG3avUnHYRkWwaflI0aGIyZVVyt6C+JZOLeHL16HNdM7UYKEyhLhGRYjR8Dz194U5mr3rVwC5Gmmexs2PZxLFc9VGCapIPHx6dMqxy7ca+wNeG+aAQEcml4QM6TA6bZA6HrNqzi+0nvY3xGYnedq76KNlqkt/2h6umDKVkG+JRIS0RKVXDDrkE5Yn3rO6cqB8+c3yUM/e9yM5TzghVSzzsUIoKaYlIVBqyh55vYrJndSf09sJXRvnM1Z/kM5+8NO81w9Q8T10fVEhLRMqvIQN6rt70RGDdsiXxPeQS/EKGUlSbRUSi0JABPVRvessWWLQITjll2nlBeeTXXbwib5lbEZEoNdQYemrcPNs+eFN607/8ZaJ3bjbtGkF7fALav1NEqqpheuiZ4+aZpvSmX38ddu0KXIKfa7hm87o1CuAiUjV1E9DzLacPCsQpnZnn/+pXie8B4+dhJz9FRCqtNgL63r3w+ONZn/7lSwd4dMtu3jV2jHcljz3a+whLzjuFc5efAMC7/2PrxHPpDPjmp8+BF/4fvJA8+NBDiaGW97xn2vnKIxeRuKqNgN7XB5/Ivm3pucmvae6bfPjNXNcP2uL67LNh3rxphzX5KSJxVRsBvbsbtm/nkR37+Nq/Pc+Rsclg2trcNOXnTCfOaeXPupcBBL726otO48IzFk1/4cknB15PeeQiElfmWXbdiUJXV5f39vYW/fru9ZsChzuazLLuHgRMlL0FBWIRqT1mttXdu/KdV1IP3cxeBg4B48BYmBuWItvE47g7BlnTEZWFIiKNoBxDLu939/1luE6g9OyVGTl64g45g7qyUESk3sV6DD0zdzzXsAokgnm24RdloYhIvSt1pagDPzWzrWa2NugEM1trZr1m1js0NFTQxbPljjdlrN5MN+4+rZqhkVjRmaqqKCJSj0oN6N3ufjbwQeCzZvZ7mSe4+wZ373L3ro6OjoIunm2Y5Jg7nVl63Kkl96nn04dhUsv0FdRFpB6VFNDdfSD5fRD4PlnSwYuVbZgklaGSra54z+pONq9bQ2d727QxdW33JiL1quiAbmbHmdmc1GPgPwNPl6thkHsziPTNKLIVw9IyfRFpJKVMii4Cvm+J8exm4H+7+4/L0qqkfIt48tUV1zJ9EWkkRQd0d38ReHcZ2xKolM0gtExfRBpJrNMWS6Vl+iLSSOo6oIO2exORxtFQOxaJiNQzBXQRkTqhgC4iUicU0EVE6oQCuohInajoBhdmNgT8pmI3jMZCILJywTVI78ckvRdT6f2YVOp78TvunrcYVkUDej0ws96oN/KoJXo/Jum9mErvx6RKvRcachERqRMK6CIidUIBvXCZbgtVAAACr0lEQVQbqt2AmNH7MUnvxVR6PyZV5L3QGLqISJ1QD11EpE4ooIuI1AkF9JDM7GQze9TMdpjZM2Z2dbXbVG1m1mRm28zs4Wq3pdrMrN3M7jWzncl/I++tdpuqxcyuTf4fedrM7jaz1mq3qZLM7DtmNmhmT6cdW2BmPzOz55Pf50dxbwX08MaAz7v7GcD5JDbFfmeV21RtVwM7qt2ImPga8GN3P53Exi8N+b6YWSfw10CXu58FNAF/VN1WVdwdwCUZx9YBj7j7acAjyZ/LTgE9JHff4+5PJB8fIvEftmELrZvZUuBS4NvVbku1mdlc4PeA2wHc/S13H65uq6qqGWgzs2ZgNjBQ5fZUlLv/HHgt4/DHgDuTj+8EeqK4twJ6EcxsGbAa2FLdllTVPwHXA8eq3ZAYOBUYAv4lOQT17eTG6Q3H3fuBrwC7gT3AG+7+0+q2KhYWufseSHQOgROjuIkCeoHM7HjgPuAadz9Y7fZUg5l9GBh0963VbktMNANnA99y99XAb4noT+q4S44NfwxYDiwBjjOzT1e3VY1DAb0AZtZCIpjf5e73V7s9VdQNfNTMXga+B6wxs3+tbpOq6lXgVXdP/cV2L4kA34guAl5y9yF3HwXuB363ym2Kg31mthgg+X0wipsooIdkZkZijHSHu3+12u2pJne/wd2XuvsyEhNem9y9YXth7r4XeMXMViQPXQg8W8UmVdNu4Hwzm538P3MhDTpBnOEh4Mrk4yuBB6O4Sd1vEl1G3cAVwHYz60seu9Hdf1jFNkl8/BVwl5nNBF4E/qzK7akKd99iZvcCT5DIDNtGg5UAMLO7gQuAhWb2KnATsB64x8yuIvGh94lI7q2l/yIi9UFDLiIidUIBXUSkTiigi4jUCQV0EZE6oYAuIlInFNBFROqEArqISJ34/1Qv8YAQ2aduAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1d8fa8da630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "model=XGBoostBaseTree(lamb=0.1,gamma=0.1)\n",
    "model.fit(data,-1*target,np.ones_like(target))\n",
    "plt.scatter(data, target)\n",
    "plt.plot(data, model.predict(data), color='r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "分别看看lambda和gamma的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1d8eb88cf60>]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAH5VJREFUeJzt3X+QHOV95/H3d38gjcSPlUBwYnEMOAQw0VlyNj4RVTm2cAXfGWyFw2XsM0VdUVGuLs4Zzoez8A+Quivk2DHOlSuuUgwxuTiOCMjCAV9sDsFx1pVxrZAM+MBFjDGwyGgFWkugQdpdfe+P6dHOznZP9/zo6Z6ez6tqa2d7erqfnZK+8+z3+T7PY+6OiIj0voGsGyAiIp2hgC4iUhAK6CIiBaGALiJSEAroIiIFoYAuIlIQiQO6mQ2a2R4zezD4+Twze8LMnjezbWZ2UnrNFBGROM300D8LPFvz8xeAO939AuAgcH0nGyYiIs1JFNDN7BzgI8DXg58N2AjcF5xyD7ApjQaKiEgyQwnP+wrweeCU4OfTgWl3nw1+fgUYDXuhmW0GNgMsX778ty666KLWWysi0od27959wN1XxZ0XG9DN7Apgv7vvNrMPVA+HnBq6hoC7bwW2AoyNjfnExETcLUVEpIaZ/SLJeUl66BuAj5rZvwGWAqdS6bGPmNlQ0Es/B3i11caKiEj7YnPo7n6zu5/j7ucC1wA73f3fAY8CVwenXQc8kForRUQkVjt16H8C/Gcz+2cqOfW7OtMkERFpRdJBUQDc/THgseDxC8D7Ot8kERFphWaKiogURFM9dBERqdixZ5Ivfu+nvDpd5uyREjddfiGb1oVWb3eNArqISJN27Jnk5u1PU56ZA2ByuszN258GyDSoK+UiItKkL37vpyeCeVV5Zo4vfu+nGbWoQgFdRKRJr06XmzreLQroIiJNOnuk1NTxblFAFxFp0k2XX0hpeHDBsdLwIDddfmFGLarQoKiISJOqA5+qchERKYBN60YzD+D1FNBFRDooy/p0BXQRKbxuBdms69M1KCoihVYNspPTZZz5ILtjz2TH75V1fboCuogUWjeDbNb16QroIlJo3QyyWdenK6CLSKF1M8hmXZ+ugC4ihdbNILtp3Sh3XLWG0ZESBoyOlLjjqjWqchER6YRuTwLKsj5dAV1ECi+Pk4DSoIAuItKmvGx2oYAuItKGrCcT1YodFDWzpWb2IzP7sZn9xMxuD45/w8x+bmZ7g6+16TdXRCRfsp5MVCtJD/0osNHd3zSzYeAHZvY/g+ducvf70mueiEi+ZT2ZqFZsD90r3gx+HA6+PNVWiYj0iKwnE9VKVIduZoNmthfYDzzs7k8ET/03M3vKzO40syURr91sZhNmNjE1NdWhZouI5EPWk4lqmXvyzraZjQDfBv4YeB34JXASsBX4mbv/aaPXj42N+cTEROutFZFCy0u1SLPSbreZ7Xb3sbjzmqpycfdpM3sM+LC7fyk4fNTM/hr4L803U0SkIk/VIs3KS517kiqXVUHPHDMrAR8CnjOz1cExAzYBz6TZUBEptjxVi/SqJD301cA9ZjZI5QPgXnd/0Mx2mtkqwIC9wH9IsZ0iUnB5qhbpVbEB3d2fAtaFHN+YSotEpC+dPVJiMiR4Z1Et0qu02qKI5EKeqkU6ZceeSTZs2cl54w+xYcvOVHZJqqWp/yKSC91aFTGqIiVJpUoz1SxZDPI2VbbYLpUtikiW6oMsVP4K+Le/Ncr9uycXHa9dyzzqtVHrnW/YsjM0hTQ6UmLXeHMZ66Rli0q5iEjfiKqk+dsfvhRbYdNsFU4Wg7wK6CLSN5oNprXnNxugs1gSQAFdRPpGs8G09vxmA3QWg7wK6CLSN8KCbJT64NtsgM5if1FVuYhI36itpAkbsKwaDalgaaUKp9tLAqjKRUT6UrNVK1lKZXEuEZGi6FbdezcpoItI38rLKomdooAuIh2V1ZrmvbqWeicpoItIx2S1pnkvr6XeSQroItIxjWZTdiqwhvXEO33fXu3tK6CLSKRmA1va092jeuL1wbyd+4bd48Zte7lh297QcsY80cQiEQlVDWyT02Wc+eDZaAnYtKe7R/XEB806dt+we1SLu5O8B1lSQBcpuFbX5G5lS7i0p7tH9bjn3Dt237hefZ63xVNAFymwVnrZVa2kT9Ke7h7V467epxP3TdKrz+u2eMqhixRYO4OFrW4Jl2Zt902XXxg6u7Oa1+7EfcPuUS+v2+Kphy5SYO0MUuZxS7huLHhVew+A+ux81u9BI+qhixRYOxsv53VqfDdmd9beo5dKGGMX5zKzpcDjwBIqHwD3ufutZnYe8PfASuBJ4Fp3P9boWlqcS6S7emEBql4KmFnp5BZ0R4GN7v4eYC3wYTNbD3wBuNPdLwAOAte302AR6bws1uRuRjuDtrJYbMrFK134N4Mfh4MvBzYCnwqO3wPcBnyt800UkXbkZQGqbszw7HeJBkXNbNDM9gL7gYeBnwHT7j4bnPIKEPrum9lmM5sws4mpqalOtFlEekxUTzxqk4m8lgXmXaKA7u5z7r4WOAd4H3Bx2GkRr93q7mPuPrZq1arWWyoiPSuqJx4lr2WBeddUlYu7T5vZY8B6YMTMhoJe+jnAqym0T6TwujkomNUAZDM97mbKAjWgulBsQDezVcBMEMxLwIeoDIg+ClxNpdLlOuCBNBsqUkTdXPa1E/dqNYBGlU/Wa2bxq0a/D+Sv3LIbkpQt/ksqg56DVFI097r7n5rZ+cyXLe4BPu3uRxtdS2WLIguD4oAZcyH/B0dHSuwa39jR+27YsjMyqCYJpO2UQIa9tp4BP9/ykca/RI2o32ekNMzR2eO5LtVsVsf2FHX3p4B1IcdfoJJPF5EY1SA+OV3GmB9wCgvmkM6gYKNrJumtt1ORUjtJKepDpdm8edTvM12eWXSsXypnNPVfJGW1FR4QUT1QJ41Bwbhrxq0i2O5a55vWjbJrfCNf+cTajiwp0KkPgCJRQBdJWVjPtpG01goJW5ulXqOg16m1zluZ7BS2BHDUWjMrlg13pJ29SGu5iKQsSc9w0Izj7m0N4MUNWLab9mi00mGzmpnsFDX4ecdVa7jjqjWLfmegY+3sNQroIimLq/DoxIBd0gqWaiCNGuBsFPSyWqyrUe5+1/jGhjn/fqtyUUAXSVlYz7Y6MNqpPSqbHbBsNThnsYxAqxtt9EMArxdbtthJKluUXLviCnjooaxbIUU0NAQzi6tvkupY2aJI33jySVi3Dq68MvbU5/Yd4v/+7HUOvz3LKUuH+J13nc5Fq0/tQiPD3f2Dn3P47dnI54cGjcsuOjPTNrbquX2HeOS5/czOzXc+e+73GehO/YkCukhVuQwbNsDttzc87UT+eWXdxJVN2U1cWblnki/GTNz5uxQmK3XDRcBzIQO+F/VhSiWOArpI1ZEjsGxZ7GlZL/naqJqlUQVLL9dh92tOvFkK6CIAc3Nw7BiUWt/xvRsBM66aZdO60cgp8f1Qh93vNLFIBODttyvfEwT0Tk2waUWjvw6q8ri5s3SHAroIVPLnkCigZxkwk/x1kPdt5yQ9SrmIQCV/Doly6FlNsIHoSUr1fx10I+estcjzRwFdBJrqoUN2g3SdnH7fjm6u4y7JKaCLQKKAnoceaZZ/HdTKutJHwimgi0BsQM9TjzQPJXzdqPTJwwdor9GgqAjE5tCTVJf0k7QrfWrXkHfmP0B37JnsyPWLSgFdBGJ76FnWnudR2pU++gBtjQK6CMQG9Cxrz/OovjRypDTM0uEBbty298QGFO3QB2hrFNBFYD6gR6RcNFlnseqWcnd+Yi1HZ49z8MhMx9Ij+gBtTeygqJm9A/gb4F8Ax4Gt7v4XZnYb8AfAVHDqLe7+3bQaKpKqag69VIpdKyVPg3R5GDiMS4+00r68lGf2miRVLrPA59z9STM7BdhtZg8Hz93p7l9Kr3kiXRL00B96/iA3/6+XG66Vkhd5qbyJSoNU29NK+/L6AZp3sQHd3fcB+4LHh83sWUDvqhRLENC/9PjLPVNfnZda8KjZq4NmbbUvbx+gvaCpHLqZnQusA54IDn3GzJ4ys7vNbEXEazab2YSZTUxNTYWdIpK9chnMePHN8E0i8jgYl9bA4Y49k2zYspPzxh9KNMAZNb4wF7EbWh7fy6JIHNDN7GTgfuAGdz8EfA14F7CWSg/+z8Ne5+5b3X3M3cdWrVrVgSaLpODIESiVOHtF+KBoHgfj0hg4bKX+O2oxsFENbHZdopmiZjZMJZh/0923A7j7azXP/xXwYCotFOmGchlKpZYG47IamExj4LDVNE5UekQDm92VpMrFgLuAZ939yzXHVwf5dYDfB55Jp4kiXRAE9GYH47IcmExj4LCTaRwNbHZfkh76BuBa4Gkz2xscuwX4pJmtBRx4EfjDVFoo0g3l8oka9GYG47IemOz0wGHS5XmT0sBmdyWpcvkBYCFPqeZciiPIoTeraDMaVf/d2zRTVAROpFyaVbQZjdrtqLdp+VwRaDmgh/VojUoufe3t38cMpo/M9FT+WGmS3qWALgKVgD4y0vTLagf+JqfLGJVBJYDp8syJ87Sjj3SDUi4i0HIOHeYXqRodKRE+laYiz8u/NjuZSPJJPXQRaDnlUivJQGgeB0vzsiaMtE89dBFYULbYqiQDoXHnZNFT1mYSxaGALgId6aGHrWlSK8mM0yy2XSta6WU/U0AXgbZy6FVhu/isWDacuPwvq55y0Uov+5ly6CIzMzA313ZAh/ZK/rLqKWsyUXGohy4Ss/1ct2TVU9ZkouJQD10kZoPobmk0SWnDlp2pTkzSZKJiUEAXqdlPNEuNJimplFCSUMpFJCc9dGg8SUmlhBJHAV0kJzn0WiollFYooIvkqIdepVJCaYUCukhOcui1ojZeVimhNKJBUZEc9tC1fZu0QgFdJIc5dFApoTRPAV0khz30ejv2TKq3LrEU0EVymEOvpeVtJanYQVEze4eZPWpmz5rZT8zss8HxlWb2sJk9H3xfkX5zRVKQ05RLlZa3laSSVLnMAp9z94uB9cAfmdm7gXHgEXe/AHgk+Fmk9+Q85aKadEkqNqC7+z53fzJ4fBh4FhgFPgbcE5x2D7AprUaKpKpchqGhylcOqSZdkmqqDt3MzgXWAU8AZ7n7PqgEfeDMiNdsNrMJM5uYmppqr7UiaejAWuhpUk26JJU4oJvZycD9wA3ufijp69x9q7uPufvYqlWrWmmjSLo6sP1cmrS8rSSV6G9MMxumEsy/6e7bg8Ovmdlqd99nZquB/Wk1UiQN1VLAz+16nvXHjB/tmcxtkFRNuiSRpMrFgLuAZ939yzVPfQe4Lnh8HfBA55snko7a/TuXzB7jrYHhruzfKZKmJD30DcC1wNNmtjc4dguwBbjXzK4HXgI+nk4TRTqvthSwNHOUt4dOOlEKWNsT1oQe6SWxAd3df0Bl45Qwl3W2OSLdUVvyt3T2GOXhJcDC3YEATeiRnpLPOi2RlJ09UmIyCOpLZ4/y5knzg6LVwL10eCByQo8CuuSRArr0lWoKpXaLt6Wzx3h92ciC88ozc4uCeZUm9EheKaBL36hfE8Wp5BKXBjn0pDShR/JKG1xI3whbE8WBUk0OvdZIaVgTeqSnqIcufSMqVbJk9hizJy1dcKw0PMhtH70E0CYT0jsU0KVv1A6E1irNHuO33z3K6EgpNHArgEuvUECXvnHT5RcuyKEDlIYGKM28zQXnnsmu8Y0Ztk6kfcqhS98IWxPlC1f8RuXJHC/OJZKUeujSsxrN4ox6btGaKAcPVr4roEsBKKBLT2q0LRs0McMz59vPiTRDAV16Uty2bIlneOZ8+zmRZiigS09qZVu20Odyvv2cSDM0KCo9KWq25oAZ3sxrFNClQBTQpSeFbcsGMOfh4dyYX0lxwZrnyqFLgSjlIj2pmguvVrIMmDUM5tVnFg2QKocuBaKALrmXpATxvPGHIl9fH+YXDJAq5SIFooAuXdHqzj+NyhNrXx81rT/KiQFSBXQpEOXQJXW1+3c680G50f6dO/ZMsmHLTm7YtrdheWJVWE69NDzIimXDodc/MUCqHLoUiAK6pC6uZrxe7QdAlPoSxLBp/XdctYZbr7yk8RK4yqFLgSjlIqmp3R0ozKvT5dBUTNgHQL2wEsRF0/prRKZ7lHKRAjGPqAw4cYLZ3cAVwH53/83g2G3AHwBTwWm3uPt34242NjbmExMTbTVYekN97jvMSGmYo7PHF65+ODwYG8yrVSujnVif/Lbb4PbbYW4OBvQHq+STme1297G485L8C/4G8OGQ43e6+9rgKzaYS3+J62WXhgcxC5+i30hYCWKjXHysI0dgyRIFcymE2H/F7v448EYX2iIF0mgKfjW/PX1kJvH1qgOcUSWILSuXlT+XwminW/IZM3vKzO42sxVRJ5nZZjObMLOJqampqNOkIKrVKVGJvNGRErvGN7Jp3WjizZbjPgAafXjEKpeVP5fCaDWgfw14F7AW2Af8edSJ7r7V3cfcfWzVqlUt3k56QVx1Sv0Gy1HT92sZxH4AJP1gCKWALgXSUkB399fcfc7djwN/Bbyvs82SXtQob17tZdcOYNaWGkapDdZRtea1HxJNO3JEKRcpjJbKFs1stbvvC378feCZzjVJ8qq2xPC00jBmMH1k5kQpYFTqo9rLDlMtNQyriqkP1vXrtzQz4zSSeuhSILEB3cy+BXwAOMPMXgFuBT5gZmupFBy8CPxhim2UHKgPuNPl+Xx2tdpkZNkwB0Py3PUpkUbLAMQF60a15i1RQJcCiQ3o7v7JkMN3pdAWybG4MsTyzBxLhgYW1ZHX97Lj1mbpaLBOolyG00/v7j1FUqLiW0kkSSXJr8ozodPva4N0s8sApE45dCkQTf2XRJKsZnj2SCm2l93K1nGpUspFCkQBPc/274eXX+76bR99bj//44e/YOrwUVadsoRr17+T//oO+Oo/v8DR2eOhr1kyNMBn1vw67N7d8NoffOtl9h8+uuj4macsiX1tKg4fVkCXwlBAz7NLL4UXXuj6bT8YfJ3w1fnjDSUYWbm70ZNfjX99KlauzOjGIp2lgJ5Xx4/Diy/CNdfApz7Vtdt+/r4f8/pbxxYdP335SfzZ1e8Jfc0Pf/Y62/e8whtvHWPl8pO4at05rH9X9EBjs+enygze//5s7i3SYQroeXXwYCWoX3opXHll1277D7sGQqftG/BnV35k0fEdeya5+UdPU149Pwv4H18f5I7fXROZS18ffIlIZ6nKJa+q696ccUZXbxs1jd6BDVt2LlrZMHdVKyJ9TAE9rw4cqHzv8vo3jdZXCVuuNndVKyJ9TAE9rzLqocetr1Lf+05lwSwRaYkCel4l6KFXl6o9b/yh0HRIqzatG2XX+EYs4vna3ncqC2aJSEs0KJpXMT30uCn0nRA1mai2953Kglki0hIF9Lw6cABOPhmWLg19utFgZLPBNGqxrJsuvzB2BUTIaA0WEVlEAT2vpqYa5s87NRiZpKev3rdIb1BAz6sDBxrmz5OkQ5KI6+mr9y3SOxTQ82pqCs48c9HhanpkcrqMwYJJQHGDkWGpFZUdihSHAnpOHXn1Nf738ZX8x/GHTgRfYEF6xOFEUB+tSYeEBe76105Ol7lx297IzZxVdijSexTQc2jHnkl+78AUr5wzhjOf1146PLAoPVIN5tUt3qJy4lGvDaOyQ5HepDr0HPrv//hjls0c5Y1lp504Vp6ZC93eDRamR6Jy4lGvrRe2KYWI9Ab10HPo6C9fA+CN0qmJzq9Nj7ST+260mbOI5J966Dn0G4OVDSBqe+gAI6Xh2FmZ7eS+lTcX6W2xAd3M7jaz/Wb2TM2xlWb2sJk9H3xfkW4z+0N1Kv/ca/uBhT300vAgt330ktg9OxstrlWrflq/8uYivS9JyuUbVPaS+ZuaY+PAI+6+xczGg5//pPPNK46o2Zi1z1cHM3+7fAiAg0EPfbTu/LD8du31TysNs3R4IDJvXr2eJgyJFEtsQHf3x83s3LrDHwM+EDy+B3gMBfRISWdjVp9feaQS0F8vnbog+N64bW/shwHAdHmG0vAgn17/a9y/ezJ06r4mDIkUT6s59LPcfR9A8H3xDJiAmW02swkzm5iqLjjVZ5JsAlE7mLmy/CtmbYBDS5efCP6T0+UFJYy1KytGXf/R56ZiUzQiUhypV7m4+1ZgK8DY2FhU6XOhJZmNWTuVf+WRQxxcdipuAwyaxS7C1ej66omL9I9We+ivmdlqgOD7/s41qXiSbAJRO5i5svwr3iidSml4kDkP/wys/zBo5r4iUkytBvTvANcFj68DHuhMc/Krnc0kGm0CUb3ujdv2smRogBXLhll55BBvnbqi4c5BUR8G9dcXkf4Rm3Ixs29RGQA9w8xeAW4FtgD3mtn1wEvAx9NsZNba3UwiahlaIHQw8+Kho5zy3ot5b/C6uDXJtcytiECyKpdPRjx1WYfbkltxg5pJAmlYLnvDlp2h152dmjqxdG7SYK1cuYho6n8CUYOO1Z56qz33sOsOHJ/jtLcOLdjcQsFaRJLQ1P8EogYXG1WgtHrd095+kwG84eYWIiJhFNCJH/CMGnRsVIGSZBA17Lpnz7xZedBg+zkRkTB9H9CrA56NJu5sWjcaOkEnqgLltNJw7DWjrjs+FgRy9dBFpEl9n0OPGvC8YdtebvvOTzCD6SMzkYORYRUoZsROBqpalB/fvr3yXT10EWlS3wf0RuuHT5fnF7cKG/CMqkC5cdvehvdquFBXdXkE9dBFpEl9H9Brp9zHCetlh1WgVDdxDrtXbE37gQOVk9VDF5Em9X0OPen64VVJdgRqNHMzdqGuqSk45RRYsiRxm0REQD30BWmTJD31RuujhK1JXp9/j0vHcOCAeuci0pK+D+gwnzapT4fUa7Q+StSa5Hd+Yu2ClExUiufEB0XNLFERkWb0bcolrE68voxwpDTMimXDidYST7LmOSRYSEs9dBFpUV/20OMGJluZZp9kzfPq9aHB2ixTU7BmTdP3FxHpjYC+cydcc03HLve7R2b4PyGzPAfvNFg23NI1nzwyEzpzdNAM7l54zU3B1wlfrnk8NaUeuoi0pDcC+llnwdVXd+xyD/7wF5HPXbv+nbGvf+HAW+x9aZq3js2y/KQh1v7aCABPvPA6s8fng/rQgPGvzj+dFWcsT964gQG4/vrk54uIBHojoF9yCfzlX7Z9mWoVyuSp4emR0ZES145vjL3Gzdufpnz+wtmhd1y1hhKLUynna5VEEemS3gjoHdBOBUutRoOfu8Y3aplbEclMYQJ6w+n0hAfiqtEmdvhJOvgpItJthQjoSbaIiwq4BuyKSbPUiq0jFxHJSM/VoYfVjyepAY8KuM0GYm3ILCJ51VMBPWrt8qgp+5PT5RNBv1OBOGptdOXORSRr5hG77qRhbGzMJyYmWn79hi07Q4P3oFnk7kEwX4UCyTZ0FhHJEzPb7e5jcee1lUM3sxeBw8AcMJvkhu2IyoPPuWNAVEhXFYqI9INODIp+0N0PdOA6oWqrVwYa9MQdGgZ1VaGISNHlusqlvnqlUVoFKsE8Kv2iKhQRKbp2B0Ud+L6Z7TazzWEnmNlmM5sws4mp6vZqCUXVjg+aRb5mzn3R4KexcIBURKSI2g3oG9z9vcC/Bv7IzN5ff4K7b3X3MXcfW9XkOt9RaZLj7oxG9LirVSfV52vTMNWqGAV1ESmitgK6u78afN8PfBt4XycaVdWodrxRGeKmdaPsGt/I6EhpUU49bI1yEZEiaDmgm9lyMzul+hj4PeCZTjUMGk/iSVIPrmn6ItJP2hkUPQv4tlXy2UPA37n7P3WkVYG4zSDiNqPQNH0R6SctB3R3fwF4TwfbEqrVHYSg0sOvX2FR0/RFpKhyXbbYrtjt3kRECqTQAR3a6+GLiPSSnlqcS0REoimgi4gUhAK6iEhBKKCLiBSEArqISEF0dYMLM5sCftG1G6bjDCC15YJ7kN6PeXovFtL7Ma/d9+Kd7h67GFZXA3oRmNlE2ht59BK9H/P0Xiyk92Net94LpVxERApCAV1EpCAU0Ju3NesG5Izej3l6LxbS+zGvK++FcugiIgWhHrqISEEooIuIFIQCekJm9g4ze9TMnjWzn5jZZ7NuU9bMbNDM9pjZg1m3JWtmNmJm95nZc8G/kUuzblNWzOzG4P/IM2b2LTNbmnWbusnM7jaz/Wb2TM2xlWb2sJk9H3xfkca9FdCTmwU+5+4XA+upbIr97ozblLXPAs9m3Yic+Avgn9z9Iiobv/Tl+2Jmo8B/Asbc/TeBQeCabFvVdd8APlx3bBx4xN0vAB4Jfu44BfSE3H2fuz8ZPD5M5T9s3y60bmbnAB8Bvp51W7JmZqcC7wfuAnD3Y+4+nW2rMjUElMxsCFgGvJpxe7rK3R8H3qg7/DHgnuDxPcCmNO6tgN4CMzsXWAc8kW1LMvUV4PPA8awbkgPnA1PAXwcpqK8HG6f3HXefBL4EvATsA37l7t/PtlW5cJa774NK5xA4M42bKKA3ycxOBu4HbnD3Q1m3JwtmdgWw3913Z92WnBgC3gt8zd3XAW+R0p/UeRfkhj8GnAecDSw3s09n26r+oYDeBDMbphLMv+nu27NuT4Y2AB81sxeBvwc2mtnfZtukTL0CvOLu1b/Y7qMS4PvRh4Cfu/uUu88A24HfybhNefCama0GCL7vT+MmCugJmZlRyZE+6+5fzro9WXL3m939HHc/l8qA105379temLv/EnjZzC4MDl0G/L8Mm5Sll4D1ZrYs+D9zGX06QFznO8B1wePrgAfSuEnhN4nuoA3AtcDTZrY3OHaLu383wzZJfvwx8E0zOwl4Afj3GbcnE+7+hJndBzxJpTJsD322BICZfQv4AHCGmb0C3ApsAe41s+upfOh9PJV7a+q/iEgxKOUiIlIQCugiIgWhgC4iUhAK6CIiBaGALiJSEAroIiIFoYAuIlIQ/x9iwna1HTSg5wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1d8eb8910f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model=XGBoostBaseTree(lamb=1,gamma=0.1)\n",
    "model.fit(data,-1*target,np.ones_like(target))\n",
    "plt.scatter(data, target)\n",
    "plt.plot(data, model.predict(data), color='r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1d8fc9e3b38>]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3W+QXOV15/HvmVFL6hFCIyFBxGAb8PLHNiQIxhSbcWWJwEACNjIb18bZYLLFRtktOzHEiy2TF5Dd9aIUjklS3nKVYmzkimNDAGMHMDaFwF5ra0lGSDZyBCHGmDCSmZHQwMCMpNHM2RfdLXX33Nt9b3ffvre7f5+qqZm5c7v7UReceeY85zmPuTsiItL5+tIegIiItIYCuohIl1BAFxHpEgroIiJdQgFdRKRLKKCLiHSJyAHdzPrNbKeZPVz8/gwze9rMXjCze81scXLDFBGReuLM0D8B7Cn7/s+Au9z9LOAgcGMrByYiIvFECuhmdhpwNfCl4vcGrAfuL96yFdiQxABFRCSaRRHv+wvgU8Dy4vcnAZPufrT4/SvAUNADzWwjsBFg2bJlF5177rmNj1ZEpAft2LFjv7uvqXdf3YBuZtcA4+6+w8wuLV0OuDWwh4C7bwG2AAwPD/vo6Gi9lxQRkTJm9vMo90WZoY8AHzSz3wSWAidSmLEPmtmi4iz9NGBvo4MVEZHm1c2hu/tn3P00dz8d+G1gm7v/R+BJ4LeKt90AfCuxUYqISF3N1KF/GvhjM/sXCjn1u1szJBERaUTURVEA3P0p4Kni1y8CF7d+SCIi0gjtFBUR6RKxZugiIlLw0M4x7vzu8+ydnOHUwTy3XHkOG9YFVm+3jQK6iEhMD+0c4zMPPsvM7BwAY5MzfObBZwFSDepKuYiIxHTnd58/FsxLZmbnuPO7z6c0ogIFdBGRmPZOzsS63i4K6CIiMZ06mI91vV0U0EVEYrrlynPI5/orruVz/dxy5TkpjahAi6IiIjGVFj5V5SIi0gU2rBtKPYBXU0AXEWmhNOvTFdBFpOu1K8imXZ+uRVER6WqlIDs2OYNzPMg+tHOs5a+Vdn26ArqIdLV2Btm069MV0EWkq7UzyKZdn66ALiJdrZ1BNu36dAV0Eelq7QyyG9YNccd15zM0mMeAocE8d1x3vqpcRERaod2bgNKsT1dAF5Gul8VNQElQQBcRaVJWDrtQQBcRaULam4nK1V0UNbOlZvYPZvYjM/uJmf1p8fo9ZvYzM9tV/Lgg+eGKiGRL2puJykWZoR8G1rv7m2aWA35oZt8p/uwWd78/ueGJiGRb2puJytWdoXvBm8Vvc8UPT3RUIiIdIu3NROUi1aGbWb+Z7QLGgcfd/enijz5rZj82s7vMbEnIYzea2aiZjU5MTLRo2CIi2ZD2ZqJy5h59sm1mg8A3gT8EDgC/ABYDW4Cfuvt/r/X44eFhHx0dbXy0ItLVslItElfS4zazHe4+XO++WFUu7j5pZk8BV7n754qXD5vZV4D/Fn+YIiIFWaoWiSsrde5RqlzWFGfmmFkeuBx4zszWFq8ZsAHYneRARaS7ZalapFNFmaGvBbaaWT+FXwD3ufvDZrbNzNYABuwC/kuC4xSRLpelapFOVTegu/uPgXUB19cnMiIR6UmnDuYZCwjeaVSLdCp1WxSRTMhStUirPLRzjJHN2zhj0yOMbN6WyClJ5bT1X0QyoV1dEcMqUqJUqsSpZkljkTdW2WKzVLYoImmqDrJQ+Cvg3180xAM7xhZcL+9lHvbYsH7nI5u3BaaQhgbzbN8UL2MdtWxRKRcR6RlhlTR/8/9erlthE7cKJ41FXgV0EekZcYNp+f1xA3QaLQEU0EWkZ8QNpuX3xw3QaSzyKqCLSM8ICrJhqoNv3ACdxvmiqnIRkZ5RXkkTtGBZMhRQwdJIFU67WwKoykVEelLcqpU0JdKcS0SkW7Sr7r2dFNBFJB0/+hH84z+mOoQNwIbVwOrihR0vwo4EXsgMbrwxgSeupIAuIi0VeTfl9dfDs8+2f4BpWLRIAV1EOkus7e5798JHPwqf/WzTr/vY7n1s/s5zHJqdP3Ztaa6PTb9xLledt7bp5+8UCugi0jK1dlNWBPSjR+G11+D00+G002K9RtBfAHfummIsvwqqSsL/x64prrrqvbH/HZ16cpICuoiEihvYIu+mPHAA3OHkk2OPJ+gvgOpfIvXGE/c1br53FzfduyuwnDFLtLFIRAKVAtvY5AzO8eBZqwVs5N2UpQPjYwb0sL8A+s1ijSfua5SKu6O8B2lSQBfpco325G7kSLjIuynHxwuf16yJNJaSsBn3nHvLttnXm9Vn+Vg8BXSRLtbILLukkW6Bkbe7lwJ6zBl62Iy79Dqt2GYfZVaf1WPxlEMX6WKRFykDNHokXKTt7qWUS8wZ+i1XnhO4u7OU125FbjvoNapl9Vg8zdBFulgzPbkT7RY4Pg59fbBqVayHtaPhVflrAFRn57N8LJ5m6CJdrJmDlxPdGj8xAatXQ3+0zofV40q6yqT8NTqphLFuQDezpcAPgCXF++9399vM7AzgG8Aq4Bngenc/kuRgRSSeWimKKBILnuPjx9ItWQ+Y7e6Y2IwoKZfDwHp3/xXgAuAqM7sE+DPgLnc/CzgIJL+vVURiSaMndyTj43DyyU0t2spCdWfoXuiv+2bx21zxw4H1wO8Ur28Fbge+2PohikgzsjLDLJ+Jf//5n9N34YVNLdrKQpEWRc2s38x2AePA48BPgUl3P1q85RUg8N03s41mNmpmoxOllW0R6SnVM/EVUwd56jUPPWQiq2WBWRcpoLv7nLtfAJwGXAy8K+i2kMducfdhdx9eE7NESUS6Q/lMPDc3y4rDb/Hq0hND789qWWDWxapycfdJM3sKuAQYNLNFxVn6acDeBMYn0vXauSiY1gJk+Yx75fQbALw2sCLw3jiLtllfUG23KFUua4DZYjDPA5dTWBB9EvgtCpUuNwDfSnKgIt0oVrvZDLxWowG0vHxy9fTrAOwfGFxwX5zmV7X+PdBdJxFFFWWGvhbYamb9FFI097n7w2b2T8A3zOx/AjuBuxMcp0jXKA+KfWbMVZ3rm9SiYNgC5E337uLO7z5fN+g18wuhvHxyVTGgH1hWOUM3YPum9U3/e27/9k84fHS+Lb8ksyZKlcuPgXUB11+kkE8XkTpKQXxscgbj+IJTdTAvSWJRsNZzRgl6zVSklG9SOml6EoDX8pUBPW7ePOzfMzkzu+Bar1TOaOu/SMLKKzwgpHqgShKLgvWes14XwWbaCEAhqG/ftJ7/fM5yAPYvO55yaWQ7fat+AXQTBXSRhAXNbGtJqldIUG+WarWCXuRe53Wcv/gw84sWsfyU1ZE3OwW1AA7rNbNyINeScXYi9XIRSViUmWG/GfPuTS3g1VuwLE97hNV/1wp6zbYROGZigr41a9j+mcsi3R6Wu7/juvO547rzF/ybgdaMswMpoIskLKxBVkk+19/0dvyoC5alXaPV95fGUSvotaxZV3Hbf1S1cvfbN62vmfNXlYuItFTQzLa0MNqqMyrjLlg2Gpxb0kagrDFXFI0etNELAbyaArpIwhJtQ1sUFtzGJmcY2bwt8PVSC3oTE3DmmZFvb6YFcK9RQBdpQNwNNkkHz1ppnczVYcdMubQsd98DVOUiElMWW77Wq2DJzMHGhw7B1FSslEtmWwBnkGboIjGl3fK11l8HtSpYMlGHXeq4GvNw6F7NicelgC4SU7MbbJpRr5plw7ohRjZvy27OeXy88FmdVxOhlItITK3aYNOIWn8dlCR6uHOzGpyhSzQK6CIxpRkwo/x1kOmcc2mGroCeCKVcRGJqRxlimKglfO3IOTfUSrc0Q1fKJREK6CINSGuRLislfA230h0fhyVLYPnydgyz5yigi0SUhdNx0vzroFzDlT6lGnSzhEfYmxTQRSJo58lC9WShhK/hSp+Jicjpliz8Au00CugiJfPzhY0vAf7q73+ET8+wtOyazxaubzhnZXvGlyFnDBh7Xw/I5a/Iw/R0+AN/8YtIC6JZ+gXaSRTQRUquuAKeeCLwR9tqPe62REaTaU29H7/3e3WfP+3NW51KAV2kZPduuOQS+NCHFvzofz/5L7xxaOHRZicuzfGxX/837Rhd5uwee53v//MEU4dmWbqoHwwOzc6xfGmOf3f2Gs4bWhH8wID3t1qam7c6mQK6SMnUFIyMwKc+teBHQ+8f4wsB1SV3XHc+9OiM8bziR1Bv9S/k+rnj/Y3XvqvDYmPqBnQzexvwVeCXgHlgi7v/pZndDvw+UCws5VZ3fzSpgYokam6ukPtdvrxur5QsLdJlYeGw3u7VRsaXlfLMThNlhn4U+KS7P2Nmy4EdZvZ48Wd3ufvnkhueSJu89RYAz74xX7dXSlZkZeGwVi/2RseX1V+gWVc3oLv7PmBf8espM9sD6F2V7jI1BcCjP5ti5uzOWIzLysJhWHqk36yp8WXtF2gniNXLxcxOB9YBTxcvfdzMfmxmXzazwNotM9toZqNmNjpR2vYrkjXFgD52NHiOk8XFuKQWDh/aOcbI5m2csekRRjZvq9vnPay3zZx7IuOTcJEDupmdADwA3OTubwBfBN4JXEBhBv/nQY9z9y3uPuzuw2vUv0Gy6s03AViyMrgyI4uLcUl0fWzk8I6wZmBDKXal7FWRqlzMLEchmH/N3R8EcPdXy37+18DDiYxQpB2KM/Rr33c2D7/cH2sxLq2FySQWDhtN44SlR7Sw2V51Z+hmZsDdwB53/3zZ9bVlt30I2N364Ym0STGgv+/CM2O1nk3zOLok2uS2Mo2T6Ta+XSrKDH0EuB541sx2Fa/dCnzEzC4AHHgJ+INERijSDsWAzvLlbDg7+mJc2guTrV44bHX9txY22ytKlcsPgaDWaKo5l+5RzKFzwgmxHtZtOxpV/93ZdGKRCFTM0ONI8zi6JChN0tm09V8Ejgf0ZctiPSxoRmsUcukX/On3MIPJ6dmO2hijNEnnUkAXgULKZdky6Iv3R2v5jsaxyRmMwqISwOTM8WZeav8q7aCUiwgUZugNHou2Yd0Q2zetZ2gwT/BWmoLy/iZZE3czkWSTZugi0FRAL4myEJrFxdKs9ISR5mmGLgKFlEvMCpdqURZC692Txky5XrdE6RwK6CLQkhl6UE+TclF2nKaxSanbSi97mQK6CLQkoFeX/A3mc6wcyEUu/0trptxtpZe9TDl0EWhJygWaK/lLa6aszUTdQzN0EWjJDL1Zac2UtZmoe2iGLgKZCOi1NimNbN6W6MYkbSbqDgroIvPzhZRLygG91iYllRJKFEq5iExPFz63IIferFqblFRKKPUooIs02JgrSSollEYooItkMKCrlFAaoYAu0mAv9CSFHbysUkKpRYuiIhmcoZcvkLb7rFLpXAroIhkM6KBSQolPAV0kgymXag/tHNNsXepSQBfJ6Ay9RO1tJaq6i6Jm9jYze9LM9pjZT8zsE8Xrq8zscTN7ofh5ZfLDFUlAxgO62ttKVFGqXI4Cn3T3dwGXAB8zs3cDm4An3P0s4Ini9yKdp5RyiXmeaLuoJl2iqhvQ3X2fuz9T/HoK2AMMAdcCW4u3bQU2JDVIkURNTUE+D4uymYFUTbpEFasO3cxOB9YBTwOnuPs+KAR94OSQx2w0s1EzG52YmGhutCJJyEBjrlpUky5RRQ7oZnYC8ABwk7u/EfVx7r7F3YfdfXjNmjWNjFEkWVNTma5wUXtbiSrS35hmlqMQzL/m7g8WL79qZmvdfZ+ZrQXGkxqkSBJKpYC3/8NPOX3G+MnOscwGSdWkSxRRqlwMuBvY4+6fL/vRt4Ebil/fAHyr9cMTSUb5+Z3LjsxwsH9JW87vFElSlBn6CHA98KyZ7SpeuxXYDNxnZjcCLwMfTmaIIq1XXgq47MgMB/MnHisFLJ8Ja0OPdJK6Ad3df0jh4JQgl7V2OCLtUV7yt+zIDP+64hSg8nQgQBt6pKNks05LJGGnDuYZKwb1ZUdmeGvx8RLAUuBemusL3dCjgC5ZpIAuPaWUQik/4m3ZkRmmFy+tuG9mdm5BMC/Rhh7JKgV06RnVPVEcMHeWzR7izcUDkZ9HG3okq3TAhfSMoJ4oS44ept/nK1IuJYP5nDb0SEfRDF16RlCq5IQjhWuz+coZej7Xz+0ffA+gQyakcyigS88oXwgtGThyCIDf/NWzeSyXDwzcCuDSKRTQpWfccuU5FTl0gJPmDwNw0XlvZ/uH1qc1NJGWUA5dekZQT5Q/vmRt4YcZ7uUiEpVm6NKxau3iDPvZgp4o3/lO4XOGuy2KRKWALh2p1rFsEGOHZ8ZPKxKJQwFdOlK9Y9ki7/AsBXSlXKQLKKBLR2rkWLbAn2mGLl1Ei6LSkcJ2a/aZ4XEeUzpPVDN06QIK6NKRgo5lA5jz4HBuHO+kWNHzfGoKFi8ufIh0OKVcpCOVcuGlSpY+s5rBvPSTBQukGT9PVCQOBXTJvCgliGdseiT08dVhvmKB9M03FdClayigS1s0evJPrfLE8scHbeuv5dgCacYPiBaJQzl0SVz5+Z3O8aBc6/zOh3aOMbJ5Gzfdu6tmeWJJUE49n+tn5UAu8PmPLZAq5SJdRAFdElevZrxa+S+AMNUliEHb+u+47nxu+8B7arfAVcpFuohSLpKY8tOBguydnAlMxQT9AqgWVIK4YFt/mdB0z9QUDKmbonQH85DKgGM3mH0ZuAYYd/fzitduB34fmCjedqu7P1rvxYaHh310dLSpAUubTE7C9HTDD39s9z7+16PPcahGYF6xNMfhufmKe5bm+ms+Bo5XraxdsZT/euk7ueq8tQ2Pk4svhssvh3vuafw5RBJmZjvcfbjefVFm6PcAXwC+WnX9Lnf/XANjk6zbswfOOw/m5xt+iquKH4m7owXPsWpVC55EJH11A7q7/8DMTk9+KJIZL7xQCOZ/8ifw9rc39BS3Pvhs6I7NwYEcV7z7FP5u9JXQe6rl+o1cfx/TRxbO3gcHcnz6qnMbGidmcM01jT1WJGOayaF/3Mw+CowCn3T3g0E3mdlGYCPA2xsMDtJmBw4UPt94I5xxRqyHHsubXxCclx4azLN9U+EgiY9v3hap1HComPe++d5dgb8ADPj0xqtjjVOkGzVa5fJF4J3ABcA+4M/DbnT3Le4+7O7Da9asafDlpK327y98PumkWA+rV51SfcBy2Pb9cgZs37SeDeuGQvu3hF0X6TUNBXR3f9Xd59x9Hvhr4OLWDktSdeAA5HKxy/lqVaeUygjLq1DKSw3DlAfrsFrz8l8SIr2soZSLma11933Fbz8E7G7dkCR1+/cXZudmFZfLSwxX5HOYweT07LFSwLDWtaVZdpBSqWH1jlBYGKyr+7fE2XEq0gvqBnQz+zpwKbDazF4BbgMuNbMLKFSPvQT8QYJjlHY7cABWr664VB1wJ2dmj/2stPNzcCDHwelZqlWnRGq1AagXrGvVmov0uihVLh8JuHx3AmORrCjN0MvU2+wzMzvHkkV95HP9NWfZ9XqzKFiLNE5b/2WhgBl6rZOASl6fmQ3cfl8epOO2ARCR6LT1Xxbav39BQI/SzfDUwXzdWXYjR8eJSDSaoUul+XnmX3uNrc9PccamR46d8FOvxDBqtYlKD0WSo4AuFR75P3vom5vj5b6Bila3QEU6ZTCfY+VALjS1EkalhyLJUcpFKmz9+2e4GjiYP16DXspxlzb4VCtVrdx87666pYQqPRRJjgK6VJh9dRyA1/InVlwPy3FHPVGonKpZRJKhlItUeGf/YQAmqwK6w7F8ejlVrYhkhwK6VPjImQPAwhk6BB8dp6oVkexQQJcKFy0rzLbzv3Ry4M+rZ9+qWhHJDgX0DlY6SLm8vLBpBw5Afz/fvf0DWMgt5bNvVa2IZIcWRTtUI4uRkZQ15grbTFQ++1bVikh2KKB3qFqLkXGDaXmzrK/83+e48IRBTqQw+67XARFUtSKSFQroHapVi5HVM/38Gwf5Zxbzys4xzb5FOowCeoeKkg6JonqmPzjzBj9bNXRspq/Zt0jnUEDvMMfO7JycwaDijM16i5FBfcirZ/SrZt7gmaXnquxQpAMpoGdUUPAFKtIjDseC+lBZOiTKY8cmZxYeuuzO4MwUBwdOVNmhSAdSQM+gsAqWpbm+BQuhpWBeOuIt7mPLLT8yTW5+jqkTBlV2KNKBFNAzKKyCJezEoPL0SNzHlhucmQLg/e97Nxcqby7ScbSxKIPi5q/L0yPN5L5Pmn4dgAsvOqvh5xCR9CigZ1BY/nown6u7K7OZ3HepMVf1eaIi0hnqBnQz+7KZjZvZ7rJrq8zscTN7ofh5ZbLD7A2lrfylCpZy+Vw/t3/wPXXP7Kx3slBJ0PP/TrExV/XxcyLSGaLk0O8BvgB8tezaJuAJd99sZpuK33+69cPrHkGVJ+WBuHoxM6yCBYK39pc//4p8jqW5Pg5OzwaOpfR81eO56PvPFW7QDF2kI9UN6O7+AzM7verytcClxa+3Ak+hgB4qSt+VoMXM8mBe60Sg6uefnJkln+vndy95Ow/sGAvcuh+4YeiB/dDXB4ODrX4LRKQNGs2hn+Lu+wCKn4N7rQJmttHMRs1sdGJiosGX62xRDoEIW8wsBf+xyZmKMz7LOyuGPf+Tz03UTdFUOHAAVq0qBHUR6TiJly26+xZgC8Dw8HB16XNPiNJ3JWwrf79Z3SZctZ4/1tb9/fuVPxfpYI1OxV41s7UAxc/jrRtS94lyCERYX/E5D/4dWP3LIM7rhiq1zhWRjtRoQP82cEPx6xuAb7VmONnVzGEStQ6BKD3vzffuYsmiPlYO5CrSI0NN/DKIvdvzwAHN0EU6WN2Ui5l9ncIC6GozewW4DdgM3GdmNwIvAx9OcpBpa/YwibA2tEDgYuZd/+GCiuet15O8ZW1u9++H97433mNEJDOiVLl8JORHl7V4LJlVb1EzSiANymWPbN5WNz8eNVg33ebWXTN0kQ6nXi4R1KtAaXTmHvWQirb0JH/zTThyRDl0kQ6m+rQIwhYXa1WgNPO8qbSuPXCg8FkzdJGOpYBO/QXPRipQoiyitmwxsxX27y98VkAX6VjmIUEpCcPDwz46Ohr/gU89Bddf3/LxAEzPzvH69GxFb3ADVgzkGCgLttOzc0wdOsrcvNPfZyxfuujY99X6rJCSrvecYc9bfU9bHDpUCOo//CGMjLT/9UUklJntcPfhevd1Rg79pJPgiisSeeonfryX6cPBvcIXL+rDgMNH5xlY0s8vD63gHSctO/bziQNvMfrSQY6WBfVFfUZ/n3H46PyC5xtY0s8HfvnUymvFj0xYsQKG6/43IyIZ1RkB/fzz4e67E3nqP9r0yIKTe8Lkc/0VW+ffAezcOcbnqypQFhztVmTABzZfXbdRl4hIIzojoCcobMt9kOqSQgiuQCkd4hz0Ws3WtIuIhOn5RdGo/cNLopwIVGuxM0qjLhGRRvT8DL18406UmXqtksKgnuST07MVaZWb790V+Nhmjo4TEQEFdOB42qQ6HVKtVklhWE/y6m38YSmeVGrPRaSr9GzKJahOfMO6oYr+4YP53IJmWWF57qiplEzVnotIV+nJGXq9hclGFifjbOOHFjTSEhGp0pMBvdZsutHAGieV0pbeLCLSc3oyoEedTYcJqiO/5cpz6ra5FRFJUk/l0Et587CNRFEWJkvpmuozPoF453eKiLRYz8zQm6lgKVcrXbN903oFcBFJTdcE9Hrb6YMCcclQjIXJZtM1IiJJ6YqAHmU7fVjANWD7pvWRX0t15CKSVR2XQw+qH49SA96qwyRURy4iWdVRAT1sQTJsy/7Y5MyxoN+qQFy9+UiLnyKSFZ1xwEXRyOZtgcG73yz09CA43vYWtKFHRDpPWw64MLOXgClgDjga5QWbEZYHn3PHILQcUVUoItILWrEo+uvuvr8FzxOovHqlr8ZM3KFmUFcVioh0u0xXuVRXr9RKq0AhmIelX1SFIiLdrtlFUQe+Z2Y7zGxj0A1mttHMRs1sdGJiItaTh9WO95uFPmbOfcHip1G5QCoi0o2aDegj7n4h8BvAx8zs16pvcPct7j7s7sNr1qyJ9eRhaZJ5d4ZCZtylqpPSz8vTMKWqGAV1EelGTQV0d99b/DwOfBO4uBWDKqlVO16rDHHDuiG2b1rP0GB+QU5dx72JSLdqOKCb2TIzW176GrgC2N2qgUHtTTxR6sG1TV9Eekkzi6KnAN+0Qj57EfC37v5YS0ZVVO8wiHp9xbVNX0R6ScMB3d1fBH6lhWMJ1MxhEOpRLiK9JNNli83ScW8i0ku6OqCDjnsTkd7RUc25REQknAK6iEiXUEAXEekSCugiIl1CAV1EpEu09YALM5sAft62F0zGaiCxdsEdSO/HcXovKun9OK7Z9+Id7l63GVZbA3o3MLPRpA/y6CR6P47Te1FJ78dx7XovlHIREekSCugiIl1CAT2+LWkPIGP0fhyn96KS3o/j2vJeKIcuItIlNEMXEekSCugiIl1CAT0iM3ubmT1pZnvM7Cdm9om0x5Q2M+s3s51m9nDaY0mbmQ2a2f1m9lzxv5F/m/aY0mJmNxf/H9ltZl83s6Vpj6mdzOzLZjZuZrvLrq0ys8fN7IXi55VJvLYCenRHgU+6+7uASygciv3ulMeUtk8Ae9IeREb8JfCYu59L4eCXnnxfzGwI+CNg2N3PA/qB3053VG13D3BV1bVNwBPufhbwRPH7llNAj8jd97n7M8Wvpyj8D9uzjdbN7DTgauBLaY8lbWZ2IvBrwN0A7n7E3SfTHVWqFgF5M1sEDAB7Ux5PW7n7D4DXqi5fC2wtfr0V2JDEayugN8DMTgfWAU+nO5JU/QXwKWA+7YFkwJnABPCVYgrqS8WD03uOu48BnwNeBvYBr7v799IdVSac4u77oDA5BE5O4kUU0GMysxOAB4Cb3P2NtMeTBjO7Bhh39x1pjyUjFgEXAl9093XAWyT0J3XWFXPD1wJnAKcCy8zsd9MdVe9QQI/BzHIUgvnX3P3BtMeTohHgg2b2EvANYL2Z/U26Q0rVK8Ar7l76i+1+CgG+F10O/MzdJ9x9FngQ+NWUx5QFr5rZWoDi5/EkXkQBPSIzMwo50j3u/vm0x5Mmd/+Mu5/m7qdTWPAKSjo/AAAAtUlEQVTa5u49Owtz918A/2pm5xQvXQb8U4pDStPLwCVmNlD8f+YyenSBuMq3gRuKX98AfCuJF+n6Q6JbaAS4HnjWzHYVr93q7o+mOCbJjj8EvmZmi4EXgf+U8nhS4e5Pm9n9wDMUKsN20mMtAMzs68ClwGozewW4DdgM3GdmN1L4pffhRF5bW/9FRLqDUi4iIl1CAV1EpEsooIuIdAkFdBGRLqGALiLSJRTQRUS6hAK6iEiX+P+FiI3o24xZCwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1d8eb891160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model=XGBoostBaseTree(lamb=0.1,gamma=100)\n",
    "model.fit(data,-1*target,np.ones_like(target))\n",
    "plt.scatter(data, target)\n",
    "plt.plot(data, model.predict(data), color='r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
